Monitor

ABSTRACT

A monitor which can detect respiration of a sleeping person without being affected by the attitude of the sleeping person or the indoor illumination light and can easily evaluate detected respiration quantitatively through image measurement. The monitor comprises means ( 1 ) for projecting a specified illumination pattern, means ( 5 ) for picking up the image of projected light continuously, means ( 7 ) for calculating inter-frame moving amount of the illumination pattern from the image of two frames acquired by he image pickup means at different times, means ( 8 ) for generating a moving amount waveform data comprising inter-frame moving amounts arranged in time series, and means ( 9 ) for detecting the movement of an object from the moving amount waveform data.

TECHNICAL FIELD

[0001] The invention of this application relates to a monitor. Moreparticularly, the present invention relates to a monitor capable ofmonitoring the body movement etc., of a sleeping person in a noncontactmanner with image measurement.

BACKGROUND ART

[0002] With the advent of an aging welfare society, the importance ofhome health monitoring for disease prevention and self care is expectedto continue to grow, and various bioinstrumentation methods have beenproposed. An aged but healthy normal person who can live an ordinarylife can be mentioned as an example of a person to be the subject ofsuch home health monitoring. However, under the present circumstances,such a person lacks the motivation to exercise health control andperform a troublesome device operation, because he/she does not needimminent health care. Therefore, a proposal has been made of variousbioinstrumentation devices capable of measuring biomedical informationunder an unconscious condition without any device operation. Asmeasurement methods employed in such bioinstrumentation devices, thereare known a method of arranging temperature sensors in a bed or a bottommattress and, from a temperature distribution, recording body movementsat the time of getting into bed, at the time of rising from bed, andduring sleep in bed, a method of deriving an electrocardiogram from anelectrically conductive sheet and a pillow cover, and a method ofmeasuring respiration and heart beat during sleep by use of a loadsensor such as a highly accurate strain gauge or a load cell.

[0003] Recently, a proposal has been made to use such bioinstrumentationtechnology as a security device applied in the safety confirmation of anaged person besides the usage of health monitoring. For example, aleaving-bed sensor has been generally sold for detecting the leaving orfalling out of bed of a person during sleep by measuring anelectrostatic capacity etc. Additionally, an apnea detector is wellknown in which an apnea state, caused by an apnea syndrome by whichbreathing ceases during sleep, is detected by measuring respiration witha pressure-sensitive sensor, and a third party can be informed of thestate (e.g., Japanese publication of unexamined patent application No.(hereinafter referred to as JP-A-) 2000-107154).

[0004] As a matter of course, respiration provides a very useful cluefor immediately knowing the health condition of a person. Respirationmeasurement performed during sleep is expected to be useful not only indetecting an apnea syndrome but also in discovering a spasmodic incidentcaused by a sudden attack during sleep. As mentioned above, a typicalrespiration detecting method during sleep is a method of detectingrespiration based on time base measured values of a load sensor or apressure-sensitive sensor. Additionally, a method of using a vibrationsensor, a radio-wave sensor, or air-pressure detection is publicly known(e.g., JP-A-H7-327939, JP-A-H11-28195, and JP-A-2000-83927). In thesemethods, since a measured signal is weak, a high-performance signalamplifier or some kind of signal processing is required to acquire anddetect a stable signal, and, as a system, it becomes expensive and largein scale.

[0005] On the other hand, some proposals have been also made of a methodof acquiring an image of a sleeping person by use of an image pickupdevice and detecting respiration based on the acquired image. Withrecent developments in electronic equipment, a high-performance imagepickup device has appeared on the market at an extremely low price, and,since the device has noncontact properties, the method of detectingrespiration based on an image has been brought to public attention as atechnique having high practical usefulness.

[0006] For example, in “Image-processing device and patient-observingdevice” of JP-A-H11-86002 and “Region-of-interest setting device ofrespiration monitoring and respiration-monitoring system” ofJP-A-H11-225997, the basic features of those inventions is to monitorrespiration by examining a difference between images acquired indifferent time by the image pickup device.

[0007] The “image-processing device and patient-observing device” ofJP-A-H11-86002 is composed of a TV camera, a respiration monitoringdevice, and a local-region automatic setting processing device. Thelocal-region automatic setting processing device is composed of an edgedetecting section that detects an edge included in a local region imageset on an image for which a patient who requires care is photographed bythe TV camera, a brightness distribution measuring section that measuresthe brightness distribution of each local partial region image dividedby the edge in the local region image, and a determining section thatsets a local region image to extract movement information by analyzinginformation about a detected edge and information about a measuredbrightness distribution. The local region image is divided into aplurality of local partial regions that are identical in brightness, anda time differentiation process is applied to each pixel included in eachlocal partial region, and the total amount of the time differentiationis calculated. A time base change of this amount is analyzed, and aperiodic appearance is detected as respiration, whereas irregularitiesin the period and amplitude are detected as great body movements, suchas a body twist.

[0008] In the “Region-of-interest setting device of respirationmonitoring and respiration-monitoring system” of JP-A-H11-225997, acalculation is first performed of the absolute value of a differencebetween every one frame of a plurality of frame images picked up by aCCD camera over ½ periods of respiration. Thereafter, the differenceimages are integrated and stored, the positions and sizes of variationregions are then calculated from variation information that has beenintegrated and stored, and they are set as temporary regions in orderfrom the largest to the smallest region in the variation ones.Thereafter, a judgment is made of whether a concentration-valuehistogram, that shows the distribution of the number of pixels of eachconcentration value, exhibits a two-peak characteristic having a heightgreater than a predetermined value in the temporary regions and whetherthe area value of the variation regions is greater than a given value.If this condition is satisfied, the temporary region is set as a regionof interest (abbreviated as ROI). Further, a time differentiationprocess is performed in the set ROI, the absolute value of thedifference between each pixel is then obtained, and a surface integralis applied. The surface integral is performed in a time series manner,and, like the invention of JP-A-H11-86002, a time base change in thissurface integral is analyzed, and a periodic appearance is detected asrespiration, whereas irregularities in the period and amplitude aredetected as great body movements, such as a body twist.

[0009] Further, a method of detecting respiration by an optical flow ofmovement of a sleeping person is known as an image-using technique otherthan the aforementioned methods. The optical flow is characterized bydetecting the movement of a sleeping person as a velocity vector, and arespiration waveform having a periodic rhythm and a body-movementwaveform having a high peak can be obtained from a vector field byemploying the fact that most upward vectors are detected in inspirationwhereas most downward vectors are detected in expiration.

[0010] The respective methods described above are to observe themovement of shadows on a quilt by use of illumination light, and thereremains the fundamental problem of being sometimes incapable ofdetecting the movement of shadows depending on lighting conditions, theposture of a sleeping person, or the design of the quilt. Additionally,since an image pickup device must be set close to the sleeping person inorder to photograph the shadow on the quilt, the respiration monitoringaccording to the aforementioned methods is considered to entail apsychologically overpowered feeling when the person to be monitored goesto bed.

[0011] Additionally, it is said that the method based on a timedifferentiation can evaluate a frequency of the movement of a targetedperson, but cannot make a quantitative evaluation of the movementthereof. In contrast, in the optical-flow method, the optical flowenables a quantitative evaluation of the movement of a sleeping person,but, in practice, much computation time is needed to calculate theoptical flow, and there remains the problem of requiring expensiveprocessing equipment.

DISCLOSURE OF INVENTION

[0012] Therefore, the invention of this application has been made inconsideration of the foregoing circumstances, and it is an object of theinvention to provide a monitor capable of detecting respiration of asleeping person without being affected by the posture of the sleepingperson or the indoor illumination light and capable of easily evaluatingdetected respiration quantitatively through image measurement.

[0013] The monitor of the present invention comprises lighting patternprojecting means for projecting a specified lighting pattern, imagepickup means for picking up light of projected wavelength continuously,moving amount calculating means for calculating an inter-frame movingamount of the lighting pattern from two frames of images acquired by theimage pickup means at different times, moving amount waveform generatingmeans for generating moving amount waveform data comprising inter-framemoving amounts arranged in time series, and waveform detecting means fordetecting a movement of an object to be picked up from the moving amountwaveform data.

[0014] The waveform detecting means is to detect the body movement andrespiration of a sleeping person, and the monitor further comprisessafety deciding means for deciding the safety of the sleeping personfrom the body movement and respiration of the sleeping person and signaloutputting means for, when the safety deciding means decides that thesleeping person is in a dangerous state, outputting a signal indicatingit.

[0015] The moving amount calculating means can easily calculate themoving amount of the lighting pattern by calculating the inter-framemoving amount of the lighting pattern in an axial direction connectingthe lighting pattern projecting means and the image pickup means.

[0016] The waveform detecting means can detect a respiratory pattern anda body movement pattern by detecting a periodic pattern as therespiratory pattern and by detecting a variation having a high peak asthe body movement pattern from moving-amount waveform data.

[0017] The safety deciding means can decide a dangerous state of asleeping person by deciding that the sleeping person is in a dangerousstate when a period of the respiratory pattern falls into disarray in ashort time, when the period of the respiratory pattern suddenly changes,when the respiratory pattern continuously disappears, or when the bodymovement pattern frequently appears in a short time.

[0018] The moving amount calculating means can easily and reliablydetect the movement of an object to be photographed by calculating aninter-frame moving amount of the lighting pattern in an axial directionconnecting the lighting pattern projecting means and the image pickupmeans while providing a positive or negative sign in accordance with amoving direction.

[0019] The waveform detecting means can reliably detect a respiratorypattern by, from among respiratory patterns, detecting a value having apositive or negative sign corresponding to a movement in a directionfrom the lighting pattern projecting means to the image pickup means onan image and a value having a positive or negative sign corresponding toa movement in a direction from the image pickup means to the lightingpattern projecting means on the image as an expiratory pattern and as aninspiratory pattern, respectively, or vice versa.

[0020] The waveform detecting means can well adapt to processing in acomputer by counting zero crosses where signs are reversed between anexpiratory pattern and an inspiratory pattern to count respirations.

[0021] The waveform detecting means can further exactly calculate arespiration number per unit time or a respiration period by calculatingthe respiration number per unit time or the respiration period from thecounted respirations.

[0022] The waveform detecting means can further easily and exactlycalculate a respiration number per unit time or a respiration period byconducting a frequency analysis of moving-amount waveform data orsampling data obtained by sampling respiratory patterns, by calculatingthe most prominent frequency from a resulting frequency spectrum, and bycalculating the respiration number per unit time or the respirationperiod from this frequency.

[0023] The waveform detecting means can further calculate therespiration number per unit time or the respiration period throughdigital signal processing by conducting a frequency analysis withdiscrete Fourier transform or discrete wavelet transform.

[0024] The safety deciding means can reliably decide that a sleepingperson is in a dangerous state by deciding whether respiration belongsto normal respiration, Cheyne-Stokes respiration, centralhyperventilation, ataxic respiration, or Kussmaul respiration fromexpiratory patterns and inspiratory patterns and by deciding that thesleeping person is in a dangerous state when the respiration belongs toany one of Cheyne-Stokes respiration, central hyperventilation, ataxicrespiration, and Kussmaul respiration.

[0025] The moving amount calculating means calculates an inter-framemoving amount in a partial region of the lighting pattern in an axialdirection connecting the lighting pattern projecting means and the imagepickup means, and determines a region in an image where an inter-framemoving amount in the partial region appears much greater than apredetermined amount or where an inter-frame movement in the partialregion appears frequently beyond a predetermined amount as a regionwhere the sleeping person exists, and the safety deciding means cansafely and reliably decide that the sleeping person is in a dangerousstate by deciding that the sleeping person is in a dangerous state whenthe region where the sleeping person exists moves in a shorter time thana predetermined time and moves frequently beyond a predetermined amount.

[0026] The moving amount calculating means calculates an inter-framemoving amount in a partial region of the lighting pattern in an axialdirection connecting the lighting pattern projecting means and the imagepickup means, and determines a region in an image where an inter-framemoving amount in the partial region appears much greater than apredetermined amount or where an inter-frame movement in the partialregion appears more frequently than a predetermined frequency as aregion where the sleeping person exists, and the safety deciding meanscan reliably decide that the sleeping person is in a state of imminentlyfalling from a bed by deciding that the sleeping person is in adangerous state when the region where the sleeping person exists isbrought closer to one end of the bed than a predetermined distance.

[0027] Additionally, it is possible to know that the sleeping person isin a dangerous state at a location remote therefrom by providinginforming means for, based on a signal output from signal outputtingmeans, informing a third party that the sleeping person is in adangerous state.

[0028] The informing means can urge a third party to swiftly deal withthe situation by informing the third party that the sleeping person isin a dangerous state by voice, characters, symbols, intensity of lightincluding interior illumination light, or vibrations through atelecommunication line.

[0029] The lighting pattern projecting means projects a lighting patternthat has a plurality of lighting spots, and the moving amountcalculating means calculates an inter-frame moving amount of eachlighting spot in an axial direction connecting the lighting patternprojecting means and the image pickup means while providing a positiveor negative sign in accordance with a moving direction, and calculatesan inter-frame moving amount of the lighting pattern by use of a movingamount of one or more of the lighting spots, whereby the moving amountof the lighting pattern can be easily calculated.

[0030] The lighting pattern projecting means projects a lighting patternthat has a plurality of lighting spots, and the moving amountcalculating means calculates an inter-frame moving amount of eachlighting spot in an axial direction connecting the lighting patternprojecting means and the image pickup means while providing a positiveor negative sign in accordance with a moving direction, and calculatesthe total moving amount of each lighting spot as an inter-frame movingamount of the lighting pattern, whereby a statistical noise influencecan be reduced.

[0031] The lighting pattern projecting means projects a lighting patternthat has a single slit ray or a plurality of slit rays, and the movingamount calculating means calculates an inter-frame moving amount of eachlighting spot in an axial direction connecting the lighting patternprojecting means and the image pickup means while providing a positiveor negative sign in accordance with a moving direction, and calculatesan inter-frame moving amount of the lighting pattern by use of a movingamount of a pixel corresponding to the single or plural slit rays,whereby the moving amount of the lighting pattern can be easilycalculated.

[0032] The lighting pattern projecting means projects a lighting patternthat has a single slit ray or a plurality of slit rays, and the movingamount calculating means calculates an inter-frame moving amount of eachpixel corresponding to a slit ray in an axial direction connecting thelighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection, and calculates the total moving amount of each pixelcorresponding to the slit ray as an inter-frame moving amount of thelighting pattern, where by the number of measurement points can beincreased, and a statistical noise influence can be reduced.

[0033] The lighting pattern projecting means is disposed directly abovea part close to one end of an edge of bedding facing a head of or a footof a person sleeping on the bedding, and the image pickup means isdisposed directly above a part close to an opposite end thereof, wherebymonitoring can be performed without allowing the sleeping person to havean intense consciousness of being monitored.

[0034] The provision of presence-in-bed detecting means for detectingthe presence/absence of a sleeping person makes it possible to swiftlyand accurately judge whether the sleeping person is absent or thesleeping person has stopped his/her breathing.

[0035] The provision of a pressure sensitive switch that can be placedunder the sleeping person and can be used to detect the presence/absenceof the sleeping person makes it possible to swiftly and accurately judgewhether the sleeping person is absent or the sleeping person has stoppedhis/her breathing.

[0036] This specification includes the contents of the descriptionand/or the drawings of Japanese Patent Application No. 2000-372820 onwhich the priority of this application is based.

BRIEF DESCRIPTION OF THE DRAWINGS

[0037]FIG. 1 is a schematic diagram showing the structure of a monitoraccording to the invention of this application;

[0038]FIG. 2 is a schematic diagram explaining the calculation of aninter-frame moving amount of a lighting pattern on an image in a movingamount calculating means;

[0039]FIG. 3 is a schematic diagram showing the principle of the monitoraccording to the invention of this application;

[0040]FIG. 4 shows an example of a waveform pattern of moving-amountwaveform data;

[0041]FIG. 5 is a schematic diagram showing the principle of therespiration detection of the monitor according to the invention of thisapplication;

[0042]FIG. 6(a), FIG. 6(b), FIG. 6(c), and FIG. 6(d) show examples ofnormal and abnormal respiratory patterns;

[0043]FIG. 7 is a schematic diagram showing a lighting pattern projectedby a lighting pattern projecting means;

[0044]FIG. 8 is a schematic diagram showing a lighting pattern projectedby the lighting pattern projecting means;

[0045]FIG. 9 is a schematic diagram showing an arrangement of thelighting pattern projecting means and an image pickup means;

[0046]FIG. 10 is a schematic diagram showing an arrangement of thelighting pattern projecting means and the image pickup means;

[0047]FIG. 11 is a schematic diagram showing an arrangement of thelighting pattern projecting means and the image pickup means;

[0048]FIG. 12 is a schematic diagram showing the structure of an indoorsafety monitor that is the invention of this application; and

[0049]FIG. 13 is a schematic diagram showing the structure of an indoorsafety monitor that is the invention of this application.

BEST MODE FOR CARRYING OUT THE INVENTION

[0050] The invention of this application has the aforementionedfeatures, and embodiments thereof will hereinafter be described.

[0051]FIG. 1 is a schematic diagram showing the structure of a monitoraccording to the invention of this application. The monitor 6 iscomposed of a lighting pattern projecting means 1, an image pickup means5, a moving amount calculating means 7, a moving amount waveformgenerating means 8, a waveform detecting means 9, a safety decidingmeans 10, a signal outputting means 11, and an informing means 12. Alighting pattern 4 is first projected from the lighting patternprojecting means 1 onto a body 2 of a sleeping person or onto bedding 3.It is preferable to set the wavelength of projected light to be infraredrays, so as to project without allowing the sleeping person to haveconsciousness of being monitored. The lighting pattern 4 projected tothe body 2 or the bedding 3 is continuously picked up as an image by theimage pickup means 5. The image pickup means 5 can pick up, for example,an infrared ray that is a wavelength of the projected light. In responseto a movement in an optical axial direction of the image pickup meansresulting from a movement of the body 2 or resulting from a movement ofthe bedding 3 responding to the movement of the body 2, a movement of alighting pattern, whose optical axis is different from that of the imagepickup means, in an pickup image plane occurs, and an inter-frame movingamount of this lighting pattern is calculated by the moving amountcalculating means 7 from two frames of images different in time thathave been acquired by the image pickup means 5.

[0052] Thereafter, in the moving amount waveform generating means 8,moving amount waveform data is generated by arranging the inter-framemoving amounts calculated by the moving amount calculating means 7 intime series. Further, in the waveform detecting means 9, the bodymovement and respiration of the sleeping person are detected from themoving amount waveform data. Even 2 or 3 frames/second of images to beacquired are enough to generate the waveform of respiration. Preferably,the images are acquired at regular intervals in order to generate themoving amount waveform data.

[0053] The monitor 6 of the invention of this application may beprovided with the safety deciding means 10 by which the safety of asleeping person is decided from the body movement and respiration of thesleeping person detected by the waveform detecting means 9. When it isdecided that the sleeping person is in a dangerous state in the safetydeciding means 10, the signal outputting means 11 outputs a signalindicating it. As a result, the dangerous state can be recorded, forexample, together with its time. Further, based on the signal output bythe signal outputting means 11, the informing means 12 automaticallyinforms a third party the sleeping person is in a dangerous state.

[0054]FIG. 2 is a schematic diagram explaining the calculation of aninter-frame moving amount of a lighting pattern on an image in themoving amount calculating means. In the moving amount calculating means7, an inter-frame moving amount of the lighting pattern in an axialdirection connecting the lighting pattern projecting means 1 and theimage pickup means 5 is calculated from two frames different in timeamong images acquired by the image pickup means 5.

[0055]FIG. 3 is a schematic diagram showing the principle of the monitoraccording to the invention of this application. In an opticalarrangement shown in FIG. 3, the following equation is established.

δ=dlZ/{h(h−Z)}

[0056] where δ: inter-frame moving amount of a lighting pattern,

[0057] h: installation heights of the lighting pattern projecting meansand the image pickup means,

[0058] d: distance between the lighting pattern projecting means and theimage pickup means,

[0059] l: focal length of the image pickup means, and

[0060] Z: displacement amount in the height direction.

[0061] That is, the inter-frame moving amount δ of the lighting patterncorresponds to a displacement amount Z in the height direction resultingfrom a movement of the body 2 of a sleeping person or resulting from amovement of bedding 3 in response to the movement of the body 2.Therefore, in the waveform detecting means 9, a waveform showing aperiodic pattern can be detected as a respiratory pattern, and awaveform showing a variation having a high peak can be detected as abody movement pattern (i.e., pattern in which the waveform is varied bya body movement, such as turning over in bed) separately from each otherin the moving amount waveform data generated by the moving amountwaveform generating means 8. In the respiratory pattern, a respirationnumber per unit time can be known from its periodicity. Further, thetime transition of the periodicity shows the stability of therespiration of the sleeping person. Further, since the inter-framemoving amount of the lighting pattern corresponds to a displacementamount in the height direction as mentioned above, it serves also as ameans for knowing the depth of the respiration.

[0062] A predominant frequency component can be calculated by conductinga frequency analysis of moving amount waveform data or respiratorypattern with discrete Fourier transform or discrete wavelet transform,and a respiration number per unit time can also be calculated from thevalue of this frequency. Fast Fourier transform can be mentioned as atypical example of discrete Fourier transform. For example, on theassumption that the most prominent sample in a result obtained byapplying fast Fourier transform to the respiratory pattern is the M-thsample in sampling data in which N samples of respiratory patterns havebeen sampled at sampling intervals of □t second, the most prominentfrequency f can be obtained by M/(□t×N), and a respiration number perminute can be obtained by 60×f. A respiration period can be obtained by1÷f. As a matter of course, various frequency analysis techniques otherthan fast Fourier transform can be applied in the invention of thisapplication.

[0063] In the safety deciding means 10, for example,

[0064] (i) when the period of a respiratory pattern falls into disarrayin a short time,

[0065] (ii) when the period of a respiratory pattern suddenly changes,

[0066] (iii) when a respiratory pattern continuously disappears, and

[0067] (iv) when a body movement pattern frequently appears in a shorttime, it is decided that a sleeping person is in a dangerous state. Thecondition of (i) or (ii) is considered to be caused by an infirmity ofthe lungs, such as spontaneous pneumothorax or bronchial asthma, acardiopathy, such as congestive heart failure, or a cerebrovasculardisease, such as cerebral hemorrhage. The condition of (iii) appearswhen respiration stops. A possible condition of (iv) is the fact thatthe sleeping person suffers from some reason and is in distress.

[0068] In the moving amount calculating means 7, preferably, aninter-frame moving amount of a lighting pattern in an axial directionconnecting the lighting pattern projecting means 1 and the image pickupmeans 5 is calculated from two frames of images different in time whileapplying a positive or negative sign in accordance with a movingdirection. Hereby, concerning the respiration, an expiratory pattern andan inspiratory pattern shown in FIG. 4 are always obtained, and therespiration can be counted by measuring the number of “zero crosses”(intersections where the sign is reversed) appearing from the expiratorypattern to the inspiratory pattern or from the inspiratory pattern tothe expiratory pattern. This is advantageous when processing isperformed by a computer.

[0069]FIG. 4 shows an example of a waveform pattern of moving-amountwaveform data. The moving amount waveform data generated by the movingamount waveform generating means 8 shows a waveform pattern, forexample, shown in FIG. 4.

[0070]FIG. 5 is a schematic diagram showing the principle of therespiration detection of the monitor according to the invention of thisapplication. In an optical arrangement shown in FIG. 5, when the heightchanges upward “A,” a lighting pattern on an image moves in a direction“A′” from the image pickup means to the lighting pattern projectingmeans. In contrast, when the height changes downward “B,” the lightingpattern on the image moves in a direction “B′” from the lighting patternprojecting means to the image pickup means on the image.

[0071] Therefore, concerning the inter-frame moving amount of thelighting pattern, if moving amount waveform data is generated from theinter-frame moving amount calculated while having a positive or negativesign in accordance with the moving direction, and if a waveform showinga periodic pattern among the moving amount waveform data is detected asa respiratory pattern, a part 41 having a sign corresponding to amovement in the direction “A′” can be detected as an expiratory pattern,whereas a part 42 having a sign corresponding to a movement in thedirection “B′” can be detected as an inspiratory pattern among thedetected respiratory patterns.

[0072] Since the moving direction of a lighting pattern follows ahorizontal or vertical direction in the picked-up image by setting thehorizontal or vertical direction of a picked-up image to coincide withthe axial direction connecting the lighting pattern projecting means andthe image pickup means, the moving amount of the lighting pattern can beeasily calculated.

[0073] Further, a respiration number per unit time (or respirationperiod) can be known by a time point where the part 41 having a signcorresponding to a movement in the direction “A′” and the part 42 havinga sign corresponding to a movement in the direction “B′” each appearonce as a single breath.

[0074] In the safety deciding means, a decision may be made as towhether the respiration of a sleeping person is a normal respiration bypre-storing each respiratory pattern concerning normal and abnormalrespirations and comparing these with a respiratory pattern of thesleeping person.

[0075]FIG. 6(a), FIG. 6(b), FIG. 6(c), and FIG. 6(d) show examples ofnormal and abnormal respiratory patterns. A normal respiratory patternto be registered is a periodic pattern shown in FIG. 6(a). On the otherhand, a respiratory pattern, such as Cheyne-Stokes respiration, centralhyperventilation, ataxic respiration, or Kussmaul respiration, that isconsidered to occur when a physiologically disorder arises in the body,is registered as an abnormal respiratory pattern. As an example, arespiratory pattern of Cheyne-Stokes respiration is shown in FIG. 6(b),a respiratory pattern of central hyperventilation is shown in FIG. 6(c),and a respiratory pattern of ataxic respiration is shown in FIG. 6(d).These respiratory patterns are obviously different in the waveform fromthe normal respiratory pattern, and a decision is made as to arespiratory pattern to which that of a sleeping person belongs on thebasis of the fact that they are different in the frequency ofrespiration, the occurrence count thereof, and the depth thereof.

[0076] When it is decided that the respiration of the sleeping personbelongs to a respiratory pattern that is considered to occur whenphysiologically disorders arise in the body, the safety deciding meansdecides that the sleeping person is carrying out abnormal respirationand is in a dangerous state. When the sleeping person displays abnormalrespiration, the informing means may inform a third party about the nameof the respiratory pattern of the sleeping person, the name of a diseaseconsidered to cause the respiration, the disease organ, the diseasepart, etc. Table 1 shows the disease name or disease part when theabnormal respiratory pattern occurs. TABLE 1 Cheyne-Stokes respirationDisorder under both-sides cerebral cortex and of diencephalon Centralhyperventilation Disorder from lower midbrain to upper pons Ataxicrespiration Disorder from lower pons to upper medulla oblongata Kussmaulrespiration Diabetic coma or uremia

[0077]FIG. 7 and FIG. 8 are each a schematic diagram showing a lightingpattern projected by the lighting pattern projecting means. Preferably,a lighting pattern projected by the lighting pattern projecting meanshas a spatially discrete distribution. For example, a plurality oflighting spots 71 shown in FIG. 7, a single slit ray 81 shown in FIG. 8,or a plurality of slit rays 82 are selected and used.

[0078] In a lighting pattern having a plurality of lighting spots, aninter-frame moving amount of each lighting spot is calculated whileadding a positive or negative sign in accordance with a movingdirection, and the total moving amount of each lighting spot iscalculated as an inter-frame moving amount of the lighting pattern.Likewise, in a lighting pattern having a single or a plurality of slitrays, an inter-frame moving amount of each pixel corresponding to theslit ray is calculated while adding a positive or negative sign inaccordance with a moving direction, and the total moving amount of eachpixel corresponding to the slit ray is calculated as an inter-framemoving amount of the lighting pattern. A statistical noise influence canbe reduced by calculating the total inter-frame moving amount whenspatially discrete lighting patterns are used.

[0079] In the moving amount calculating means, a region in an imagewhere the inter-frame moving amount of a lighting pattern appearsgreatly or where an inter-frame movement of the lighting pattern appearsfrequently is determined as a region where the sleeping person exists.When this region frequently moves in a short time, the safety decidingmeans may decide that the sleeping person is in a dangerous state. Thiscan be regarded as a condition where the sleeping person suffers fromsome reason and is in distress.

[0080] The safety deciding means may decide that the sleeping person isin a dangerous state when the region where the sleeping person exists isbrought extremely close to one of the sides of a bed and when thesleeping person on the bed is being monitored. This can be regarded as asituation where the sleeping person is in a dangerous position as if tofall from the bed.

[0081] In the monitor of the invention of this application, preferably,a range where the lighting pattern is projected is set within a rangecovering positions that can be occupied by the belly, chest, back, andshoulders of the sleeping person. Likewise, preferably, a range of aregion photographed by an image pickup device is set within a rangecovering positions that can be occupied by the belly, chest, back, andshoulders of the sleeping person.

[0082]FIG. 9 and FIG. 10 are each a schematic diagram showing anarrangement of the lighting pattern projecting means and the imagepickup means. For example, as shown in FIG. 9, the lighting patternprojecting means 91 is disposed directly above a part close to the head92 or the foot 93 of the sleeping person, and the image pickup means 94is disposed directly above a part around the belly 95 of the sleepingperson. At an edge 104 of bedding 101 placed under the sleeping personfacing the head 92 or the foot 93 of the sleeping person as shown inFIG. 10, the lighting pattern projecting means 106 may be disposeddirectly above a part close to one end 105 of the edge 104, and theimage pickup means 108 may be disposed directly above a part close to anopposite end 107 of the edge 104. The arrangement shown in FIG. 9 cancapture the respiration of the sleeping person extremely sensitively,and has the advantage of being able to detect a detailed respiratorypattern. However, since the image pickup device comes into view of thesleeping person, there can be a case in which the arrangement isaccompanied by a psychologically oppressive feeling. In contrast, anarrangement shown in FIG. 10 cannot specifically catch the respirationunlike the arrangement shown in FIG. 9, but this arrangement isconsidered to have a weaker psychologically oppressive feeling. Ofcourse, the arrangement shown in FIG. 9 can be formed to sacrificeslightly the sensitivity of respirations as in FIG. 11, and anoppressive feeling may be reduced by disposing the lighting patternprojecting means 111 and the image pickup means 112 at one side end 113of the bedding. These arrangements are to be appropriately selecteddepending on the situation.

[0083] Any type of lighting pattern projecting means can be used if thelighting pattern projecting means can project light spatiallydiscretely. Use can be made of, for example, a fiber grating, adiffraction grating, a lens array, a formed image of a light sourcearray or an aperture array, a device that collimates an outgoing beam ofthe light source array or the aperture array, etc.

[0084] In the monitor of the invention of this application, as anauxiliary means for accurately detecting whether a sleeping person ispresent or absent, a pressure sensitive switch may be disposed in thebedding placed under the sleeping person. The presence/absence of thesleeping person is determined by ON/OFF of this switch.

[0085] Further, in the monitor of the invention of this application, theinforming means has a voice outputting function and informs a thirdparty by voice that the sleeping person is in a dangerous state. It mayinform the third party by characters, symbols, intensity of lightincluding interior illumination light, or vibrations. Further, theinforming means may have a function to connect to a telecommunicationsline, such as a general telephone line, ISDN line, PHS line, or cellulartelephone line, and may inform the third party by voices, characters, orsymbols that the sleeping person is in a dangerous state.

[0086]FIG. 12 is a schematic diagram showing an arrangement of thelighting pattern projecting means and the image pickup means. Theprinciple of the invention of this application is not applied limitedlyto a monitor for a sleeping person, of course. For example, as a matterof course, the principle can be applied also to an indoor safety monitorthat monitors the safety of a person in a room. The indoor safetymonitor has the same structure as the monitor of the invention of thisapplication, and is shown, for example, in FIG. 12.

[0087] First, a lighting pattern 123 is projected from a lightingpattern projecting means 121 to a body 122 of a person to be monitored.The lighting pattern 123 projected to the body 122 is continuouslypicked up as an image by an image pickup means 124, and, from two framesof images different in time acquired by the image pickup means 124, aninter-frame moving amount of the lighting pattern 123 resulting from amovement of the body 122 is calculated by the moving amount calculatingmeans.

[0088] Thereafter, in the moving amount waveform generating means,moving amount waveform data is generated by arranging the inter-framemoving amounts calculated by the moving amount calculating means in timeseries. Further, in the waveform detecting means, a body movement andrespiration of the person to be monitored are detected from the movingamount waveform data.

[0089] The indoor safety monitor is provided with a safety decidingmeans. From the body movement and respiration of the person detected bythe waveform detecting means, the safety deciding means decides whetherthe person to be monitored is safe or not. When the safety decidingmeans decides that the person is in a dangerous state, the informingmeans automatically informs a third party the person to be monitored isin a dangerous state.

[0090] In the moving amount calculating means, an inter-frame movingamount of a lighting pattern on an image in an axial directionconnecting the lighting pattern projecting means 121 and the imagepickup means 124 is calculated from two frames different in time amongimages acquired by the image pickup means 124.

[0091] In the waveform detecting means, a waveform indicating a periodicpattern is detected as a respiratory pattern, whereas a waveformindicating a variation having a high peak is detected as a body movementpattern among the moving amount waveform data generated by the movingamount waveform generating means.

[0092] The safety deciding means decides that a time zone where abody-movement pattern of the person to be monitored disappears indicatesa state where the person stops moving, and, in this time zone, thesafety deciding means decides that the person is in a dangerous statewhen the period of the respiratory pattern falls into disarray in ashort time, when the period of the respiratory pattern suddenly changes,or when the respiratory pattern continuously disappears.

[0093] The indoor safety monitor mentioned above works effectivelyespecially when a monitoring region is limited. For example, this isextremely adaptable for monitoring a person having a bath as shown inFIG. 13. Likewise, this works effectively to monitor the safety of aperson in a bathroom.

[0094] The present invention is not limited to the aforementionedembodiments.

[0095] It is permissible to project light that has been subjected toamplitude modulation and extract a modulating signal from a picked-upsignal. Thereby, it is possible to reduce an influence by which generallight, such as indoor illumination light, is changed into a noise.

[0096] If precision is sacrificed to some degree, the frame may be afield. Therefore, the frame set forth in the appended claims has a broadconcept including a field.

Industrial Applicability

[0097] As described in detail in the foregoing description, according tothe invention of this application, it is possible to provide a monitorby image measurement capable of reliably detecting the respiration of asleeping person and easily making a quantitative evaluation of thedetected respiration without being influenced by the posture of thesleeping person, illumination light, and design of a quilt.

[0098] The invention of this application is strongly expected to be putinto practical use as a technique that can support the achievement ofswift emergency treatment when a senior citizen or sick person fallsinto a crisis condition.

[0099] All publications, patents, and patent applications cited in thepresent specification are to be employed in the present specification asreferences without any changes.

What is claimed is:
 1. A monitor comprising: lighting pattern projectingmeans for projecting a specified lighting pattern; image pickup meansfor picking up light of a projected wavelength continuously; movingamount calculating means for calculating an inter-frame moving amount ofthe lighting pattern from two frames of images acquired by the imagepickup means at different times; moving amount waveform generating meansfor generating moving amount waveform data comprising inter-frame movingamounts arranged in time series, and waveform detecting means fordetecting a movement of an object to be picked up from the moving amountwaveform data.
 2. The monitor as set forth in claim 1, wherein thewaveform detecting means detects a body movement and respiration of asleeping person, the monitor further comprising: safety deciding meansfor deciding the safety of the sleeping person from the body movementand respiration of the sleeping person; and signal outputting means for,when the safety deciding means decides that the sleeping person is in adangerous state, outputting a signal indicating it.
 3. The monitor asset forth in claim 2, wherein the moving amount calculating meanscalculates an inter-frame moving amount of a lighting pattern in anaxial direction connecting the lighting pattern projecting means and theimage pickup means.
 4. The monitor as set forth in claim 2 or 3, whereinthe waveform detecting means detects a periodic pattern as a respiratorypattern, and detects a variation having a high peak as a body movementpattern from the moving amount waveform data.
 5. The monitor as setforth in claim 4, wherein the safety deciding means decides that thesleeping person is in a dangerous state when a period of the respiratorypattern falls into disarray in a short time, when the period of therespiratory pattern suddenly changes, when the respiratory patterncontinuously disappears, or when the body movement pattern frequentlyappears in a short time.
 6. The monitor as set forth in claim 4 or 5,wherein the moving amount calculating means calculates an inter-framemoving amount of the lighting pattern in an axial direction connectingthe lighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection.
 7. The monitor as set forth in claim 6, wherein from amongrespiratory patterns, the waveform detecting means detects a valuehaving a positive or negative sign corresponding to a movement in adirection from the lighting pattern projecting means to the image pickupmeans on an image and a value having a positive or negative signcorresponding to a movement in a direction from the image pickup meansto the lighting pattern projecting means on the image as an expiratorypattern and as an inspiratory pattern, respectively, or vice versa. 8.The monitor as set forth in claim 7, wherein the waveform detectingmeans counts zero crosses where signs are reversed between an expiratorypattern and an inspiratory pattern and counts respirations.
 9. Themonitor as set forth in claim 8, wherein the waveform detecting meanscalculates a respiration number per unit time or a respiration periodfrom the counted respirations.
 10. The monitor as set forth in claim 1or 7, wherein the waveform detecting means calculates a respirationnumber per unit time or a respiration period by conducting a frequencyanalysis of the moving-amount waveform data set forth in claim 1 or thesampling data obtained by sampling respiratory patterns set forth inclaim 7, by calculating the most prominent frequency from a resultingfrequency spectrum, and by calculating the respiration number per unittime or the respiration period from this frequency.
 11. The monitor asset forth in claim 10, wherein the waveform detecting means conducts afrequency analysis with discrete Fourier transform or discrete wavelettransform.
 12. The monitor as set forth in any one of claims 7 through9, wherein the safety deciding means decides whether respiration belongsto normal respiration, Cheyne-Stokes respiration, centralhyperventilation, ataxic respiration, or Kussmaul respiration fromexpiratory patterns and inspiratory patterns and decides that thesleeping person is in a dangerous state when the respiration belongs toany one of Cheyne-Stokes respiration, central hyperventilation, ataxicrespiration, and Kussmaul respiration.
 13. The monitor as set forth inclaim 2, wherein the moving amount calculating means calculates aninter-frame moving amount in a partial region of the lighting pattern inan axial direction connecting the lighting pattern projecting means andthe image pickup means, and determines a region in an image where aninter-frame moving amount in the partial region appears much greaterthan a predetermined amount or where an inter-frame movement in thepartial region appears frequently beyond a predetermined amount as aregion where the sleeping person exists, and the safety deciding meansdecides that the sleeping person is in a dangerous state when the regionwhere the sleeping person exists moves in a shorter time than apredetermined time and moves frequently beyond a predetermined amount.14. The monitor as set forth in claim 2, wherein the moving amountcalculating means calculates an inter-frame moving amount in a partialregion of the lighting pattern in an axial direction connecting thelighting pattern projecting means and the image pickup means, anddetermines a region in an image where an inter-frame moving amount inthe partial region appears much greater than a predetermined amount orwhere an inter-frame movement in the partial region appears morefrequently than a predetermined frequency as a region where the sleepingperson exists, and the safety deciding means decides that the sleepingperson is in a dangerous state when the region where the sleeping personexists is brought closer to one end of the bed than a predetermineddistance.
 15. The monitor as set forth in any one of claims 2 through14, comprising informing means for, based on a signal output from signaloutputting means, informing a third party that the sleeping person is ina dangerous state.
 16. The monitor as set forth in claim 15, wherein theinforming means informs the third party that the sleeping person is in adangerous state by voices, characters, symbols, intensity of lightincluding interior illumination light, or vibrations through atelecommunications line.
 17. The monitor as set forth in any one ofclaims 1 through 16, wherein the lighting pattern projecting meansprojects a lighting pattern that has a plurality of lighting spots, andthe moving amount calculating means calculates an inter-frame movingamount of each lighting spot in an axial direction connecting thelighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection, and calculates an inter-frame moving amount of the lightingpattern by use of a moving amount of one or more of the lighting spots.18. The monitor as set forth in any one of claims 1 through 16, whereinthe lighting pattern projecting means projects a lighting pattern thathas a plurality of lighting spots, and the moving amount calculatingmeans calculates an inter-frame moving amount of each lighting spot inan axial direction connecting the lighting pattern projecting means andthe image pickup means while providing a positive or negative sign inaccordance with a moving direction, and calculates the total movingamount of each lighting spot as an inter-frame moving amount of thelighting pattern.
 19. The monitor as set forth in any one of claims 1through 16, wherein the lighting pattern projecting means projects alighting pattern that has a single slit ray or a plurality of slit rays,and the moving amount calculating means calculates an inter-frame movingamount of each lighting spot in an axial direction connecting thelighting pattern projecting means and the image pickup means whileproviding a positive or negative sign in accordance with a movingdirection, and calculates an inter-frame moving amount of the lightingpattern by use of a moving amount of a pixel corresponding to the singleor plural slit rays.
 20. The monitor as set forth in any one of claims 1through 16, wherein the lighting pattern projecting means projects alighting pattern that has a single slit ray or a plurality of slit rays,and the moving amount calculating means calculates an inter-frame movingamount of each pixel corresponding to a slit ray in an axial directionconnecting the lighting pattern projecting means and the image pickupmeans while providing a positive or negative sign in accordance with amoving direction, and calculates the total moving amount of each pixelcorresponding to the slit ray as an inter-frame moving amount of thelighting pattern.
 21. The monitor as set forth in any one of claims 1through 20, wherein the lighting pattern projecting means is disposeddirectly above a part close to one end of an edge of bedding facing ahead or a foot of a person sleeping on the bedding, and the image pickupmeans is disposed directly above a part close to an opposite endthereof.
 22. The monitor as set forth in any one of claims 1 through 21,comprising presence-in-bed detecting means for detecting thepresence/absence of a sleeping person.
 23. The monitor as set forth inany one of claims 1 through 22, comprising a pressure sensitive switchthat can be placed under the sleeping person and can be used to detectthe presence/absence of the sleeping person.